A 3D production pipeline for special effects in TV and film

This contribution describes techniques for 3D modelling and sensing of dynamic studio scenes and different visualisation techniques for the production of special effects in an improved production pipeline. The main concept of this new production pipeline is the use of a shared 3D database that is used and edited through the different production stages. The benefit of this concept was demonstrated during different phases of an experimental production. In particular during the on-set phase, the system provides visualisation tools including a preview of the composed programme for the director or an immersive feedback system for the actor. The immersive feedback is implemented using viewdependent projection onto a special retro-reflective chroma-keying cloth and does not interfere with the shape capturing sub-system. Finally the generated dynamic 3D models were used in post-production to achieve new effects, like full optical integration of real actors with virtual backgrounds (shadow casting and reception) and fully free selectable camera viewpoint. This document was originally published in Proc. of Mirage 2005 conference, Computer Vision/Computer Graphics Collaboration Techniques and Applications. March, 1-2 2005, INRIA Rocquencourt, France Additional key words: 3D reconstruction, special effects, Film and TV production, mixed reality, on-set visualisation